In Azure machine learning,
a pipeline is a type of workflow, is a workflow of machine learning tasks in
which each task is implemented as a step. These steps can be arranged sequentially
or in parallel, enabling you to build sophisticated flow logic to
orchestrate machine learning operations. Each step can be run on a specific compute
target, making it possible to combine different types of processing as
required to achieve an overall goal. A pipeline can be executed as a process
by running the pipeline as an experiment. Then each step in the pipeline runs on
its allocated compute target as part of the overall experiment run. And Azure machine learning
pipeline consists of one or more steps that perform tasks. There are many kinds of steps supported
by Azure machine learning pipelines, each with its own specialist purpose and
configuration options. Common kinds of steps in an Azure
machine learning pipeline include PythonScriptStep, runs
a specified Python script. DataTransferStep, uses Azure data factory
to copy data between data stores. DataBrickStep, runs a notebook script or
compiled jar on a data bricks cluster. AdlaStep, runs a use sequel job
in Azure Data lake analytics. And ParallelRunStep, runs a Python script as a distributed
task on multiple compute nodes. It's important to note that for
a full list of supported step types, see
Azure.pipeline.stepspackagedocumentation. You can find a link to this document from
the additional readings at the end of this lesson. To create a pipeline,
you must first define each step and then create a pipeline
that includes the steps. The specific configuration of each step
depends on the step type, for example, you can use a code that defines two Python
script steps steps to prepare data and then train a model. After these steps are defined,
you can assign them to a pipeline and run it as an experiment.